Real-Time Stereo Vision Based Fingertip Detection and Tracking

Fingertip detection and tracking is the foundation of many hand-based natural Human Computer Interaction (HCI) applications. Previous methods either can hardly provide a natural way of interaction, or are too complex to be adopted in practical systems. In this paper, we propose a real-time stereo vision-based fingertip detection and tracking method that works very well in real environment. Our method can get the accurate fingertip position regardless the finger shape for the single finger. Fingertip detection is formulated as a graph theory problem, and can be solved easily given the 2D finger skeleton image. The accurate 3D location of the fingertip can be obtained based on stereo vision. To achieve the robustness against noise and occlusion, Kalman filter is applied to smooth the trajectory of fingertip. Our method is simple yet effective, and can run in real-time on a normal PC. Keywords-fingertip detection; finger tracking; stereo vision; finger segmentation; finger skeleton; trajectory; Kalman filter